Testing Structural Properties in Textual Data: Beyond Document Grammars

This article describes research carried out in the project "Secondary information structuring and comparative discourse analysis" (SEKIMO), which is part of the research group "Texttechnological modeling of information" and is funded by the German Research Council (DFG). In our project, we use XML document grammars, i.e. DTDs (Bray et al., 2000), XML Schema (Thompson et al., 2001) and Relax NG (Clark and Murata, 2001) to formalize and interrelate linguistic phenomena in typologically diverse languages. The document grammars differ in what they describe, that is morphosyntactic structures, semantic relations and discourse functions, and in the granularity of the description; i.e. there are language or dialogue type specific document grammars on the one hand and document grammars of a more general kind on the other hand. At the level of secondary information structuring, we interrelate the document grammars, sometimes creating 'intermediate' document grammars in order to connect the specific and general levels of linguistic description. All document grammars are developed on the basis of and applied to dialogue and text corpora in different languages. (For more information about the project, see www.text-technology.de). Schema languages usually define grammatical constraints on document structures, i.e. hierarchical relations between elements in a tree-like structure. Especially but not only for the linguistic phenomena we want to describe, it seems useful to complement the concept of hierarchical validation with a methodology for defining and applying other structural constraints as there are several limitations in implementing appropriate document grammars. The main benefits of this methodology are: • Addition of constraints which are hard to express using schema languages • Independent formulation of constraints; adding new constraints does not require changes to document schema • Classification of information items; assigning classes based on fulfillment of constraints.